Screen sharing involves displaying all or part of what is displayed on a first screen on one or more other screens. Screen sharing may include displaying user actions, such as the movement of a mouse or what is typed on the screen. Screen sharing may further involve giving control of the first screen to another device, such as by giving over control of the mouse. Screen sharing is commonly used for videoconferencing, training, and customer service applications.
A customer service agent may converse with a customer through a video-based screen sharing application. During such a conversation, a customer may present views of his screen that contain information that is useful for resolving the customer's request. The customer service agent may switch her attention away from the desktop sharing application to search for online tools that would use information obtained from reading the screen for purposes such as research or recording.
Text recognition (also known as text extraction or Optical Character Recognition (OCR)) is the conversion of typed or handwritten characters to machine-encoded text. Text recognition is commonly used to process scanned documents, or to make a document searchable by a computer. There are several methods of text recognition, such as pixel-based, density-based, and fixed-position text extraction.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.